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STATEMENT OF WORK 


PROBLEM STATEMENT 

LI SCOPE 

This document reports the results of a cooperative research program between Western Aerospace Laboratories, Inc. and the National 
Aeronautics and Space Administration, titled "EVALUATING COMPONENTS OF THE T-NASA SYSTEM" The period of 
performance covers August 1995 to October, 1996. 

BACKGROUND 

One of the goals of NASA's Terminal Area Productivity (TAP) program is to increase the efficiency of surface operations, so that 
more airplanes can move in and out of airports in a given time period. Two major elements of the program are to reduce runway 
occupancy time and roll-out time by 20%, and to maintain VFR levels of performance in IFR (down to CAT IIIB [300 ft RVR]) 
visibility. 

The only way to realize the second element is to augment the pilot's situation awareness during taxi operations. Currently, situation 
awareness depends on what the pilot can see out of the windows, his or her experience with the airport, and Jeppesen paper maps. 
Under low visibility conditions, the out-the-window scene is inadequate to support the operational objectives of the TAP program. To 
take just one example, report number 464 in the ASRS data base on weather encounters documents a case where a pilot was 
attempting to taxi under unexpectedly low visibility conditions (heavy fog). He reported his forward speed as "very slow", and he 
confused the blue taxiway edge lights for the green taxiway center lights (the two colors became indistinguishable when reflected 
through the fog). He attempted to follow what he thought were the center lights, and guided the aircraft off the runway and onto a soft 
shoulder. He eventually brought the aircraft back onto the runway and to the gate, but in a time frame that falls far short of the TAP 
objectives. 


PLAN FOR RESEARCH 

The goals of the TAP program mandate a system that supplements the visual information available to the pilot in low visibility 
conditions. The LOVLASO team has developed a traffic, navigation, and situation awareness (T-NASA) system that incorporates an 
electronic map and head-up display (HUD) symbology. The research performed in FY 1996 was designed to evaluation of the separate 
and joint effect of the two components (i.e., the HUD and the taxi map) on ground taxi performance. A full report of the simulation 
follows. 

ABSTRACT 

We report the results of a part-task simulation evaluating the separate and combined effects of an electronic moving map display and 
newly developed HUD symbology on ground taxi performance, under moderate- and low-visibility conditions. Twenty-four 
commercial airline pilots carried out a series of 28 gate-to-runway taxi trials at Chicago O'Hare. Half of the trials were conducted 
under moderate visibility (RVR 1400 ft), and half under low visibility (RVR 700 ft). In the baseline condition, where navigation 
support was limited to surface features and a Jeppesen paper map, navigation errors were committed on almost half of the trials. 
These errors were virtually abolished when the electronic moving map or the HUD symbology was available; in addition, compared to 
the baseline condition, both forms of navigation aid yielded an increase in forward taxi speed. The speed increase was greater for the 
HUD than the electronic moving map, and greater under low visibility than under moderate visibility. Ihese results suggest that the 
combination of electronic moving map and HUD symbology has the potential to greatly increase the efficiency of ground taxi 
operations, particularly under low-visibility conditions. 

introduction 

One of the primary tasks facing the pilot of an aircraft is navigation, the process by which the aircraft is guided from the departure 
point to the correct destination. Currently, two navigation aids are available to pilots of glass cockpit aircraft: the typical horizontal 
situation indicator or "HSI" display and, for aircraft equipped with Head-Up Displays (HUD), a head-up guidance system. 



As currently engineered, however, neither the HS1 nor the head-up guidance system supports the task of navigating on the airport 
surface. For ground operations, navigation support is still limited to paper maps and surface features, such as signs and taxiway lights. 
Wayfinding using paper maps is a highly demanding task [1], particularly at large, complex airports [2]. The difficulty is exacerbated 
still further under reduced visibility or instrument meteorological conditions (IMC). The archives of NASA's Aviation Safety 
Reporting System contain numerous incidents triggered by pilots becoming disoriented while taxiing under low-visibility conditions. 
The consequences of disorientation include deviating from the cleared route, increased interaction with ATC, and conflicts with other 
traffic. In extreme cases, route deviations can seriously disrupt the traffic pattern across the entire airport. For example, a misplaced 
aircraft recently forced the closing of a runway for several hours at Chicago O'Hare [3]. It is clear from these incidents that surface 
navigation, particularly under low visibility, poses a major challenge to the efficiency of terminal area operations [4], 

Over the next decade, air traffic in the United States is expected to increase by 50%. In response, NASA and the FAA are 
developing a number of advanced technologies to increase the traffic-handling capacity of existing airports. One such effort aims to 
increase the efficiency of surface operations, particularly under low-visibility conditions. Here at NASA-Ames, we are currently 
developing an integrated system of navigation aids with the goal to enable pilots to achieve the level of taxi performance normally 
seen under visual meteorological conditions (VMC) in IMC [5], 

From a human factors perspective, the first step in designing such a system is to identify the information requirements of the 
ground taxi task [2], In particular, we are interested in identifying the information that is acquired from the out-the-window scene; we 
can then design displays to provide this information when it is not otherwise available (i.e., in IMC). 

Two components of the navigation task are typically distinguished, each relying on a different set of visual cues. The local 
guidance component [1], makes use of flow field characteristics, edge rate information, and the geometric relation between the focus 
of optical expansion (a point in the visual field from which the optical flow field appears to radiate, indicating the aircraft's current 
heading; [6]). These optical cues control various actions (e.g., steering, throttle adjustments, braking) that move the aircraft through 
the environment at a certain heading and a certain forward speed. The second task component, navigation awareness [1], involves the 
identification of relevant landmarks (e.g., signage, taxiway intersections, etc.) in order to maintain awareness of the aircraft's current 
location with respect to the cleared route. Navigation awareness is necessary to ensure that the pilot makes correct decisions regarding 
where and when to turn the aircraft. 

ELECTRONIC MO VING-MAP DISPLA YS 

Recent efforts to assist the pilot with ground navigation have focused on electronic moving map (EMM) displays, which depict the 
current location of the aircraft on the airport surface and update the location in real time. If the EMM also contains a depiction of the 
cleared route, a quick glance at the display is sufficient to assess the current position of the aircraft relative to the cleared route, 
allowing the pilot to maintain a high level of navigation awareness. For our purposes, an additional important feature of the EMM is 
that it provides navigation awareness without having to recognize landmarks in the out-the-window scene. Furthermore, by rotating 
the map so that it is always aligned with the current heading of the aircraft, the EMM removes the need to cognitively "align" the ego- 
referenced information in the out-the-window scene with the world-referenced information on standard paper maps [1]. Thus, not 
only is information regarding navigation awareness provided by an EMM, it is provided in a form that greatly reduces the information 
processing demands on the pilot [7], 

Given these characteristics, one would expect ground taxi performance to improve when an EMM is introduced into the flight 
deck, particularly under low-visibility conditions. A number of recent simulation studies have examined ground taxi performance 
with and without an EMM [7] [8][9] [10]. Consistent with expectations, pilots did indeed make fewer navigation errors and taxi at a 
greater speed when an EMM was present, compared to a paper-map only condition. These benefits were more pronounced in low than 
in high visibility. Further, the benefits of the EMM were even greater when graphical route guidance was provided relative to a pure 
positional display [7] [11]. 

Electronic moving maps are not without their drawbacks, however. The typical EMM provides only a 2-D plan view of the airport 
surface. This form of display can be used for local guidance at low-zoom levels, by keeping the ownship symbol aligned with the 
route guidance line or taxiway centerline (assuming the EMM includes these features). However, a 2-D plan view is very dissimilar to 
the out-the-window scene [9], where the cues that control local guidance are normally extracted. Thus, the standard EMM format is 
not well suited to display local guidance information. In addition, several simulation studies have found that EMM usage increases 
head-down time [7]; see also [12], Ground taxi requires continuous interrogation of the out-the-window scene for traffic incursions 
and other potential hazards, so any display that brings the pilot's eyes into the cockpit raises serious safety concerns [2], 

HEAD- VP DISPLA YS 

These considerations suggest that further improvements in ground taxi performance could be obtained with a display that better 
supports local guidance (cf. [1 3][ 1 4]. Wickens and Prevett [15] argue that the ideal local guidance display would provide the same 



optical cues, in the same ego-referenced forward-view perspective, that are normally extracted from the out-the-window scene. If 
such a display was also head-up, local guidance could be achieved in an "eyes-out" mode, just as it is normally. Guided in part by 
these considerations, we recently developed a candidate set of HUD symbology to support ground taxi [16][17]. Shown in Figure 1, 
the symbology set includes a series of evenly-spaced tiles overlaying the taxiway centerline, and a regularly-spaced series of cones 
stretching along each side of the cleared taxiway. These symbols are "scene-linked" such that, as the aircraft proceeds through the 
environment, the symbols undergo the same optical transformations as if they were actual objects placed on the taxiway [16]. 
Subjectively, the symbology resembles a virtual taxiway lighting system, except that the lights outline only the cleared route. Thus, 
the HUD symbology solves both shortcomings associated with an EMM: It provides a natural set of optical cues to support local 
guidance, and does so in a form that allows pilots to remain eyes-out. 

Just as the EMM provides some support for local guidance, so too does the candidate HUD symbology provide some support for 
navigation awareness. By displaying the cleared route as the highlighted taxiways on the HUD, the HUD symbology torms.a virtual 
path through the environment, yielding a "preview" of the distance to upcoming turns and the actual turns themselves. Navigation 
accuracy can be achieved by simply following the virtual path. Similar to the EMM, the HUD eliminates the need to process real- 
world landmarks and cognitively reference them to a paper map. However, given the narrow field of view on the HUD, the "preview" 
of the cleared route is maximal on sections that feature long straight- aw ays. Since the symbology overlays the cleared route directly, 
the length of the virtual path diminishes in curved areas, and disappears completely on sharp turns. Thus, our HUD symbology does 
not support (and was not designed to support) navigation awareness to the same extent as an EMM. 

THE PRESENT STUD Y 


The present research had three primary goals. One was to assess the benefits to ground taxi performance provided by the taxiway 
HUD symbology. Another was to compare directly the performance benefits associated with an EMM and the 
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Figure 1: Depiction of HUD taxi symbology over a generic taxiway. All HUD symbology shown in white (actually green). 


taxiway HUD. The third goal was to compare the magnitude of the separate benefits provided by the two forms of navigation aid 
against the benefits when both aids were available simultaneously. These comparisons are critical for establishing the usefulness of 
HUD symbology in an integrated system of ground navigation aids. EMM's are relatively inexpensive to integrate into current glass 
cockpits, so their introduction into the next generation of commercial aircraft is virtually assured [2], HUDs are much more expensive 
devices. Thus, a critical cost/benefit comparison is between the size of the performance benefit found with an EMM alone, compared 
to an EMM plus the HUD symbology. 

An additional aim of the present research was to determine whether the benefits associated with the different configurations of 
navigation displays would increase as visibility was reduced. Such a pattern follows naturally from the fact that pilots can use out-the- 



window cues to navigate under high visibility conditions. Under low visibility, however, the normal out-the-window scene cues 
become unavailable or sharply degraded, conceptually forcing greater reliance on the electronic aids. 

In addition to these purely quantitative issues, there is considerable theoretical interest in the form of the performance benefit 
associated with the two kinds of navigation aids. As we have seen, the HUD was designed primarily to support local guidance, 
whereas the EMM was designed primarily to support navigation awareness. A priori, it might be expected that EMM's would have a 
larger effect on route-following accuracy than on aspects of performance that are controlled by local guidance cues, such as forward 
speed. The opposite pattern of benefits would be predicted for the HUD symbology. 

Finally, we expected the results of the simulation to provide additional insight into the nature of the ground taxi task. For example, 
Batson et al. [8] used root mean square deviations from taxiway centerline as the primary measure of route following accuracy. From 
the pilot's perspective, however, ground taxi may not be conceptualized as a tracking task, with the goal to follow the centerline of the 
taxiway as closely as possible. Taxiways are wide enough to permit considerable deviations from centerline and still remain within 
safe taxiing parameters. Thus, our simulation provided an opportunity to determine whether route navigation accuracy, rather than 
absolute deviation from centerline is, in fact, a more appropriate error metric for the task. 

In addition to the empirical measures, various forms of subjective data were collected. Since our electronic navigation aids remove 
the need to engage in various cognitive processes associated with a paper map [1], we expected the navigation displays to reduce pilot 
workload considerably. To quantify the expected reduction, pilots rated the workload on each trial. We were further interested in 
whether workload reduction would vary by the type of navigation aid available (EMM vs. HUD vs. both). 

After completing the simulation, each participant filled out an extensive questionnaire. The questions provided an opportunity to 
solicit pilot feedback on the two navigation aids, and to record pilot opinions on the comparative efficacy of the two displays. 

PARTICIPANTS 

Twenty-four highly-experienced male airline pilots participated in the study (mean hrs of flight time > 10000). Twenty-two were 
currently occupying the position of Captain, and two of First Officer. Current aircraft type was distributed across B737, B747, B757, 
B767, MD80, and DC- 10. Twenty-two of the 24 participants reported either high or moderate levels of experience with ground taxi. 
In addition, 22 participants reported some experience flying in and out of Chicago O'Hare. Although many participants had logged a 
high number of hours in simulators, simulator experience with ground taxi was quite low (median = 3 hrs). 

THE SIMULATION 

Participants viewed a highly detailed out-the window scene of Chicago O'Hare from a simulated eye height of 16 feet (i.e., the eye 
height of the B737 flight deck). The visual scene was driven by an SGI Onyx Reality Engine 2, rear-projected on an Electrohome 
screen measuring 2.43 m (width) by 1.83 m (height). The screen image had a resolution of 640 by 512 pixels, and was updated at a 
rate of 30 Hz. Participants were seated in a chair approximately 2.43 m from the screen. At this distance, the screen provided 
approximately 53 deg of horizontal visual angle. 

The HUD consisted of a semi-transparent silvered glass sheet (combiner) measuring 24 cm in height by 20.4 cm in width. The 
combiner was oriented at an angle of 41 degrees down with respect to the observer, located approximately .30 m from the eye point, 
and centered with respect to the wide screen. The HUD symbology was generated by an SGI Personal IRIS at an update rate of 8-12 
Hz (depending on scene complexity), drawn on an XKD CRT monitor (1280 by 1024 pixels of resolution). The monitor symbology 
was projected through a Fresnel lens and reflected into the participants' eyes through the combiner glass. All symbology was green 
and appeared at a focal distance of 2.43 m. This ensured that the HUD symbology appeared at the same optical distance as the image 
on the wide screen. A portable metal frame housed and physically supported the CRT monitor and the combiner glass. On blocks of 
trials where the HUD symbology was withheld, the frame was moved to one side of the room. 

The HUD symbology is shown in Figure 1. The cleared route was depicted by vertical side cones on each side of the commanded 
taxiway, as well as rectangular tiles overlaid on the taxiway centerline. Both forms of symbol were repeated every 15.2 meters down 
the cleared route. Turn "countdown" warnings took the form of three rows of tiles beginning with the centerline tile and extending to 
the right side of the taxiway (see Figure 1). The initial row, located 46 m from the corner, contained 4 tiles (including the centerline 
tile), the second row (30.5 m away) contained 3 tiles, and the third row (15.2 m away) contained 2 tiles. The sharpness of the turn was 
depicted by a "turn sign" located just beyond the taxiway. The angle of the arrow drawn on the turn sign was veridical with the angle 
of the turn. Finally, location and ground speed information were given in a non-scene-linked triangular "Past/Present/Future" format. 
The central reference identified the taxiway/runway currently occupied by the aircraft; the lower left reference identified the taxiway 
intersection just passed, and the lower right reference identified the crossing taxiway at the next intersection. Ground speed was 
displayed digitally directly above the central reference. 

The forward route was drawn according to the following algorithm. Each route was divided into segments whose length varied 
according to the curvature of the route at that point. As the plane taxiied along the route, the symbology outlining the upcoming route 




segment was drawn on the CRT. At least 300 m beyond the participant's eye point was drawn at all times. On long, straight sections 
of the route, an upcoming segment could be drawn well before the ownship reached the end of the current segment, extending the 
route overlay well beyond the 300 m minimum. 

Note, however, that although a minimum of 300 m of route-overlay symbology was always generated by the computer, the 
symbology was not always visible on the combiner glass. Since the symbology overlaid the cleared route only, it remained in view 
only as long as the airplane was aligned with the cleared route. If the pilot committed a navigation error, and strayed off the cleared 
route, taxiways/runways in the forward field of view were not highlighted by HUD symbology. Furthermore, since the flight deck of 
the B737 is considerably forward of the plane's center of gravity, the cleared taxiway (and overlaid HUD symbology) was often out of 
range of the forward field of view just prior to sharp turns. 

The EMM, shown in Figure 2, was driven by an IBM personal computer equipped with a Pentium processor, and displayed on a 
23 -cm diagonal CRT located below and to the left of the participant at a distance of approximately 1 m. The display consisted of a 2- 
D track-up plan-view depiction of Chicago O'Hare airport that could be viewed at one of five zoom levels. The pilot could adjust the 
zoom level at any time by rotating a dial located below the right armrest. In Figure 2, the ownship symbol can be seen aligned with 
the route guidance line. The EMM was updated at a rate of 6-8 Hz, depending on the complexity of the display. Additional details 
concerning the EMM design can be found in [7]. 

The vehicle model emulated the handling characteristics of a B737. A combination of rudder and toe brakes were located below 
the forward display area. The throttle was located on the surface of the right-side arm rest, and the nose wheel tiller, a joystick, was 
located on the left-side armrest. Leftward steering was controlled by rotating the joystick counterclockwise, rightward steering by 
rotating the joystick clockwise. 

The simulation facilities were housed in a dedicated, darkened room. An experimenter station was located in a room adjacent to 
the simulation facility. The station included three computer monitors, one displaying the current out the-window view, one displaying 
HUD symbology, and one displaying the EMM. Regardl ess of w hat n avi gation a i d 
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Figure 2: Electronic Moving Map, showing the ownship proceeding along the inner (Alpha) taxiway at Chicago O'Hare. The 
ownship is following a cleared route starting in the ramp area adjacent to Concourse F and finishing at departure runway 14R. Gray 
areas were green on map; the background in the left-hand clearance area and the route overlay line were blue. 


combination of aids, were available to the pilot, all three displays were available to the experimenter. This allowed constant 
monitoring of the pilot's current location and progress. In addition, two-way communication between pilot and experimenter was 
available at all times via an intercom. 

METHOD 

The experiment contained 28 trials, each consisting of a specified route that began in a ramp area adjacent to a terminal, and 
finished when the airplane turned onto the departure runway. The routes averaged 1.7 nmi in length and took approximately 7 min to 
complete. To avoid duplication, all major terminals and runways were utilized for route construction. The assignment of routes to 
trials was random, and the same random order was maintained for each participant. 

The experiment was divided into two blocks of 12 trials each. In the baseline block the only navigation aid provided was a 
Jeppesen paper map of Chicago O'Hare. In the navigation aid-enhanced block, pilots assigned to the "EMM" group were provided 
with the paper map and the EMM; pilots assigned to the "HUD" group were provided with the paper map and the HUD taxi 
symbology; pilots assigned to the "EMM + HUD” group were provided with the paper map and both the EMM and the HUD taxi 
symbology. For half of the participants, the first block of trials formed the baseline condition, and the second block formed the 
navigation-aid enhanced condition; for the remaining participants, this order was reversed. 

Of the twelve trials in each block, the first two were considered practice, and were not analyzed. In addition, considering the trials 
sequence as a whole, trials 3,7, 16, and 22 contained an unexpected traffic conflict in the form of another airplane positioned directly 
on the cleared route. Pilots were instructed to be on the alert for a conflict at all times. When a conflict was discovered, the 
instructions were to come to a complete stop at a safe distance from the other aircraft. The other aircraft then taxiied out of the way, 
and the pilot proceeded along the cleared route. The purpose of these conflicts was to keep the pilot's attention on the outside world as 
much as possible; this was desirable given that, with the exception of these occasional conflicts, no other aircraft were present in the 
simulation. Data from these trials were not analyzed. This left 16 experimental trials (8 from each block) on which data was collected 
and analyzed. 

Within each block, half of the trials were performed under RVR 700 ft, and half under RVR 1400 ft. The two visibility levels were 
simulated by the method developed by Torres and Hoock [18], a "first-principles" calibrated fog model. However, the clarity of 
objects in the out-the-window view was compromised by the update rate (30 Hz) and the resolution of the rear-projection screen 
system. Hence, the actual RVR values of 700 ft and 1400 ft are likely overestimates of the visual range within which surface features 
such as taxi signs could be read. In an attempt to compensate for the display quality, all surface signs in the O'Hare database were 
scaled at 1.5 times actual size, to approximate realistic readability. The assignment of visibility level to trial was determined 
randomly, with the constraint that half of the experimental trials in each block were performed under low visibility, and half under 
moderate visibility. The same random assignment of visibility level to trial was repeated for each participant. 

Following the second block of trials, all pilots completed 5 additional trials with both the EMM and the HUD symbology provided. 
In this way, every participant had some experience taxiing with both navigation aids, and could call on this experience for evaluation 
purposes. 

PROCEDURE 

Each participant was randomly assigned to one of the three groups (EMM, HUD, or EMM + HUD). Upon arrival, the participant 
was seated in the simulation facility, and the various physical components of the simulator were explained. He then read a detailed 
description of the experimental procedure. The instructions informed the participant that he would be piloting a simulated B737 
through a series of gate-to-runway departure sequences. He was asked to conform as closely as possible to his normal taxi behavior 
when carrying passengers but, within those constraints, to proceed as rapidly and as accurately as possible to the departure runway. 
No maximum taxi speed was defined for the simulation. In the event that the pilot became lost or disoriented, he was told to first try 
to return to the cleared route using every means (e.g., paper map, local exploration of environment) available. If he was unable to 
return to the cleared route on his own, he was told to contact "ground control" (i.e., the experimenter) for navigation instructions. 

A minimum of a half hr break separated the first and second blocks of trials. At the completion of the experiment, all participants 
completed a detailed questionnaire recording their opinions on various features of the two navigation displays, and responded to a 
number of questions comparing the efficacy of the navigation displays. 

Each trial contained the following sequence of events. The experimenter, seated at the control facility, asked the participant 
whether he was ready for the next trial through the intercom. Following a "ready" acknowledgment, a written clearance was presented 
in the left margin of the CRT containing the EMM (note that the clearance appeared regardless of whether the EMM itself was 
present). Figure 2 shows the clearance for a route starting adjacent to Concourse F and finishing on Runway 14R. The experimenter 



repeated the clearance verbally, and solicited a "clearance received and understood" acknowledgment from the pilot. The pilot then 
followed the cleared route to the departure runway, at which point the forward screen went blank. The pilot then completed workload 
ratings for the trial (see below). Once the ratings were completed, the experimenter initiated the next trial. 

PERFORMANCE MEASURES 

Taxi performance was evaluated using a variety of empirical and subjective measures. Forward taxi speed was sampled at a rate of 
2 Hz. These values were averaged to arrive at a mean taxi speed for each trial. In addition, two measures of total trial time were 
calculated. One measured the elapsed time from the appearance of the clearance to the arrival at the destination runway. The other 
measured from when the forward velocity of the aircraft first exceeded .5 kts to the arrival at the destination runway. Since the throttle 
speed was always zero at the start of a trial, the difference between these measures gave an indication of the time taken by the pilot to 
plan the route. 

To evaluate route following accuracy, three "occupancy zones" were designated, each encompassing a certain region on either side 
of the taxiway centerline. Zone 1 included the area 2 m on either side of centerline; Zone 2 included the area from the Zone 1 
boundary to 1 1 .2 m to either side of centerline (corresponding to the width of a standard taxiway at Chicago O Hare), Zone 3 included 
areas beyond the Zone 2 boundary. Starting with the appearance of the trial clearance, and finishing when the aircraft reached the 
departure runway, the position of the aircraft was sampled at a rate of 2 Hz. Each position was coded as either Zone 1, 2, or 3. Then, 
the total number of samples for each zone was converted into the total time spent in each zone (1 sample - .5 s). These values were 
divided by the total trial duration (measured from the appearance of the trial clearance to the arrival of the airplane on the destination 
runway), yielding the measure: proportion of total trial time that the airplane was located in each zone. 

We developed the zonal approach because of concerns over the suitability of the standard accuracy metric, absolute or root mean 
square deviation from centerline [8]. The absolute deviation metric assumes that ground taxi is viewed by the pilot as a tracking task, 
with the goal to minimize deviations 

from centerline. This assumption may well be unwarranted. Taxiways provide a relatively large paved area around the centerline, 
giving pilots considerable leeway to deviate from centerline and still perform the task perfectly well. Furthermore, from the 
standpoint of navigation awareness, it is unlikely that a pilot who is currently taxiing, say, 3 m from the centerline has any less 
awareness of the airplane's location with respect to the cleared route than a pilot who is taxiing on the centerline. 

Given these considerations, we did not classify Zone 2 occupancy as inherently poorer taxi performance than Zone 1 occupancy, as 
dictated by the standard centerline deviation metric; rather, the relative amount of time spent in Zones 1 and 2 was simply taken to 
indicate pilot tolerance for small-to-moderate deviations from centerline. By contrast, Zone 3 occupations were expected to reflect 
true navigation errors; thus, the proportion of total trial time that the airplane was occupying Zone 3 formed our primary measure of 
task accuracy. These assumptions were for the most part verified by post-experimental inspection of trial performance (see below). 

SUBJECTIVE PERFORMANCE MEASURES 

Earlier, we mentioned that the electronic navigation aids used here not only provide information relevant to navigation, but do it in 
such a way that task difficulty should be reduced [7], To quantify this hypothesis, NASA TLX workload ratings [19] were collected 
from each participant. These ratings include six constructs (mental demand, physical demand, temporal demand, performance, effort, 
and frustration), each represented by a 12 point Likert scale. Following the completion of each trial, the pilot was asked to rate the 
trial on all six constructs, by marking a location along the scale. The marked location was converted to a value between 0 and 1 1. The 
values on the six constructs were then averaged, producing a single "workload" value for each trial. 

Finally, each pilot completed a lengthy post-experiment questionnaire. The questionnaire gave the pilots an opportunity to evaluate 
various features of the EMM and HUD symbology, and to assess their relative usefulness, 

RESULTS AND DISCUSSION 

After two pilots had completed the simulation, it was discovered that airport signage critical to one of the ioutes had not been^ 
included in the visual database of Chicago O'Hare. For both pilots, the affected trial was part of the baseline block. The database was 
upgraded, and the two trials were excluded from analyses. 

VISIBILITY EFFECTS 

Given our interest in potential interactions between navigation aid-related performance benefits and visibility, the first step was to 
determine whether the manipulation of visibility affected pilot performance. Accordingly, we took the baseline condition for all three 
pilot groups and analyzed a variety of dependent measures across the two levels of visibility and the two presentation orders (baseline 



block first or enhanced block first). This latter variable was included to detect the presence of any unwanted main effects or 
interactions due to block order. 

With respect to forward taxi speed, pilots taxiied at a mean value of 14.32 kts under low visibility and 15.44 kts under high 
visibility. The effect of visibility was highly reliable F(l, 22) = 18.24, p <.01, with 21 of 24 pilots showing the effect. Neither the 
main effect of block order nor the interaction of block order with visibility approached significance, both Fs < 1. A similar analysis 
revealed that the proportion of time spent in Zone 3 (i.e., the navigation error zone) was .134 under low visibility, compared to .10 
under moderate visibility, F( 1, 22) = 4.93, p <.05. Again, no effects involving block order approached significance. And thirdly, the 
mean workload rating was 6.2 for low-visibility baseline trials, compared to 5.4 for moderate visibility baseline trials. The increase in 
perceived workload as visibility was reduced was highly significant, t{ 23) = 3.53,/? < .01, In summary, when navigation aids were 
limited to a paper map, taxi under low visibility was slower, more error-prone, and more demanding than taxi under moderate 
visibility. 

NA VIGA TION AID PERFORMANCE BENEFITS 

Figure 3 shows the mean occupancy time in Zones 1, 2 and 3 as a function of visibility and condition (baseline vs. navigation aid- 
enhanced). Comparing Zone 3 occupancy between baseline and navigation aid-enhanced blocks, we see that, regardless of visibility, 
the presence of an advanced navigation aid virtually abolished Z mi 3 occupancy time. This pattern held equally for the EMM group, 
the HUD group, and the EMM + HUD group. In other words, mute deviations large enough to signal the likely presence of a 
navigation error were almost nonexistent when either or both navigation displays were present. These results support earlier studies 
showing that an EMM improves route following accuracy [7]{8], In addition, they show that the HUD symbology was just as 
effective as the EMM in improving accuracy. 

The second noteworthy aspect of these results is the effect of visibility on the proportion of time spent in Zones 1 and 2. As shown 
in the figure, Zone 1 occupancy was greater in low visibility than in moderate visibility, whereas Zone 2 occupancy was greater in 
moderate than in low visibility. This pattern held in both the baseline condition and the navigation aid-enhanced condition, across all 
three pilot groups. Clearly, pilots tolerated larger deviations from taxiway centerlines under moderate visibility than under low 
visibility. 

In our view, these findings confirm our earlier suspicion that, within a certain range, deviation from centerline is not an 
appropriate measure of task accuracy. Compared to the low 



ZONE 

Figure 3: Mean proportion of total taxi time spent in Zones 1, 2, or 3 as a function of condition (baseline vs. navigation aid- 
enhanced) and visibility (low vs. moderate). 




visibility condition, the moderate visibility condition produced faster taxi speed, reduced Zone 3 occupancy, and lower workload. 
From these other metrics, it is clear that greater deviations from centerline (as long as they are not too extreme) can be asspciated with 
better, not worse, taxi performance. 

One possible account of this behavior is as follows. Suppose that in moderate or high visibility, local guidance makes use of a 
variety of peripheral visual cues, particularly taxiway edges, in addition to the taxiway centerline. In low visibility, these peripheral 
cues are either degraded or lost entirely [5], forcing the pilot to rely more on the centerline. The forced reliance on centerline might 
influence the pilot to treat ground taxi as more of a centerline tracking task. Such a strategy would naturally yield higher occupancy in 
Zone 1 (and lower occupancy in Zone 2), relative to a higher visibility condition. 

We expected intrusions into Zone 3 to indicate bona-fide navigation errors, and thus, that Zone 3 occupancy would be a sensitive 
measure of route-following accuracy. To verify these assumptions, the position coordinates for each trial were recorded, so that each 
trial could be played back in a moderately fast forward mode. Intrusions into Zone 3 occurred on 1 1 1 baseline trials (58% of the 
baseline total). Inspection of these trials indicated that Zone 3 intrusions were the product of three relatively distinct patterns. The 
first pattern occurred when pilots took a "short cut" across the apron area, or deliberately executed a very wide turn in order to 
maximize the distance available on the cleared runway. These, apparently deliberate, deviations from centerline did contaminate the 
strongest version of our original assumption, which was that all intrusions into Zone 3 indicate a genuine navigation error. However, 
on trials where this pattern was observed, the proportion of total trial time spent in Zone 3 was only .03, and 1 1 of the 24 pilots made 
no intrusions of this sort at all. Thus, we can conclude that the vast majority of Zone 3 occupancy time was indeed the result of bona 
fide navigation errors. 

Excluding deliberate course deviations, there remained 87 trials (46% of the total) in which a Zone 3 incursion was recorded. 
Some of these were characterized by the pilot failing to turn onto a cleared taxiway, or turning onto an incorrect taxiway or runway, 
and then returning the aircraft to the cleared route with the first set of navigation decisions following the error. This pattern suggests 
that the navigation error resulted from a lack of local navigation-related features, such as taxi signs, or insufficient attention to signage 
available. However, pilots clearly retained some situation awareness regarding the relative locations of the aircraft and the cleared 
route. These "local" navigation errors occurred on 52 trials (27% of the baseline trials). On the remaining 35 trials, the navigation 
decisions following the initial course deviation did not return the aircraft immediately to the cleared route. This pattern suggests that 
the pilot had temporarily lost all awareness of the spatial relation between his ownship position and the cleared route. 

As might be expected, the time course of the two forms of error were quite different. On trials where a "local" error was 
committed, the mean proportion of total trial time spent in Zone 3 was only .16, and the average trial completion time was 7.1 min. 
That is, the aircraft remained in Zone 3 for approximately 1 min. On trials where navigation awareness appeared to break down 
completely, the proportion of total trial time spent in Zone 3 rises to .35, and trial completion time rises to 9.7 min. Thus, on these 
trials, the aircraft strayed from the cleared route for an average of 3 min. 

TAXI SPEED 

Table 1 shows taxi speed as a function of condition and visibility, as well as the difference between the baseline condition and the 
navigation aid-enhanced condition for the three pilot groups. The most important aspect of these results is that all three groups of 
pilots taxiied more rapidly in the navigation aid-enhanced condition than in the. baseline condition. The increase in speed ranged from 
1.5 kts (EMM group,, moderate visibility) to 4.5 kts (EMM + HUD group, low visibility). The mean of these speed increases was 
significantly different from zero, F(l, 21) = 22.98, p <.01. In addition, while the EMM yielded a modest increase in speed at both 
visibilities, the speedup due to the HUD symbology was considerably larger in low than in moderate visibility. To assess these effects 
statistically, each participant's speed in the baseline condition was subtracted from his speed in the navigation-aid enhanced condition. 
The resulting difference scores were then entered into an analysis of variance (ANOVA) with pilot group (Map, HUD, and EMM + 
HUD) as a between-subjects variable and visibility (moderate versus low) as a within-subject variable. The ANOVA revealed a main 
effect of visibility (greater benefits in low than in moderate visibility, F(l,21 = 12.69, p <01), and a significant interaction between 
group and visibility, F(l, 21 = 4.5, p <05). An individual comparison showed that in low visibility, the 4.5 kt increase in speed for the 
EMM + HUD group was significantly higher then the 1.5 kt increase in the EMM group, F(l, 14) = 6.51,/) <.05. 

So far, we have established that the advanced navigation aids virtually eliminated navigation errors and increased taxi speed. One 
would naturally expect that, combined, these two effects would substantially reduce trial completion time. Table 2 shows the mean 
completion time for each group of pilots in each condition and visibility level. As shown in the table, the navigation aids did indeed 
yield large reductions in trial completion time relative to the baseline condition; the time savings ranged from 1.9 min to 3.1 min. The 
mean of these time savings was significantly higher than zero, F(l,21) = 45.87, p < .01. In addition, the time savings were larger in 
low visibility than in moderate visibility, F( 1,21)= 10.87,/? <.01. 

Table 1 . Mean Taxi Speed (kts) as a function of Pilot Group and Visibility Level 



Visibility 


RVR 700 ft RVR 1400 ft 


Baseline 

13.8 

14.9 

EMM 

15.3 

16.6 

Difference 

1.5 

1.7 

Baseline 

15.5 

17.3 

HUD 

19.4 

18.9 

Difference 

3.9 

1.6 

Baseline 

12.1 

12.9 

EMM + HUD 

16.6 

15.7 

Difference 

4.5 

2.8 


Table 2. Mean Route Completion Time (min) as a function of Pilot Group and Visibility Level 


Visibility 

RVR 700 ft RVR 1400 ft 


Baseline 

7.8 

7.2 

EMM 

5.9 

5.2 

Difference 

1.9 

2.0 

Baseline 

. 7.1 

5.8 

HUD 

4.6 

4.5 

Difference 

2.5 

1.3 

Baseline 

8.4 

7.5 

EMM + HUD 

5.3 

5.3 

Difference 

3.1 

2.2 


In low visibility, the 3.1 min time savings in the EMM + HUD group was noticeably larger than the 1.9 min savings in the EMM 
group, although high variance in the EMM + HUD group kept the comparison from reaching statistical significance. This pattern can 
be understood from the fact that for both the EMM group and the EMM + HUD group, some time savings were achieved by the 
elimination of navigation errors; in addition, under low visibility the EMM + HUD group achieved additional savings from the HUD- 
related increase in forward taxi speed (Table 1 ). 

PLANNING TIME 

Recall that two values for route completion time were calculated, one from when the clearance was provided, the other from when 
the pilot started to move the aircraft. The difference represents the time taken to plan the route before actually setting out. Once 
again, it is of interest to note the impact of navigation aids on planning time, and whether the impact varied across pilot groups. 

Including all 24 participants, the mean planning time in the baseline condition was 72 s. This was reduced to 35 s in the 
navigation-aid enhanced conditions. All participants showed the reduction. Thus, as expected, pilots spent less time planning their 
route when an advanced navigation aid was present. Table 3 shows the size of the reduction separately for each pilot group and each 
visibility level. The only clear pattern in these data is that reductions in planning time in the advanced-aid condition was larger under 
moderate visibility (mean = 42 s) than under low visibility (mean = 34 s); F( 1,21)=" 4.44,/? <.05. 



This small effect is nevertheless interesting, for it is the only variable in which the benefits ot the advanced navigation aids were 
more pronounced in moderate visibility than in low visibility. In the baseline condition, planning time was slightly longer on RVR 
1400 ft trials (76 s) than on RVR 700 ft trials (68 s). This difference was eliminated when an advanced navigation aid was provided. 
One account of this pattern is that, in moderate visibility, pilots had some preview of landmarks along the beginning of the route. 
Suppose this information was incorporated into the pilot's "mental map" of the route. This process would have taken a small amount 
of time. On low-visibility trials, where the landmarks were not visible, there was obviously no effort to process them, so planning 
time was reduced accordingly. When a navigation aid was provided, we assume that all forms of route planning were reduced or 
dispensed with, thereby attenuating any systematic effects in the baseline condition. 

Table 3. Navigation- A id related reductions in planning time (s) as a function of pilot group and visibility. 

Visibility 
RVR 700 ft 

EMM 34 

HUD 33 

EMM+ HUD 35 


RVR 1400 ft 
40 
35 
50 


SUBJECTIVE MEASURES 

WORKLOAD RATINGS - Consistent with earlier analyses, workload ratings are shown separately for each group of pilots and 
each visibility level in Table 4. As shown in the table, pilots rated the workload on navigation aid-enhanced trials much lower than the 
workload on baseline trials; the average size of the workload reduction was 25% under low visibility and 19/6 under moderate 
visibility. The workload difference between the navigation-aid enhanced conditions and the baseline condition was significantly 
greater than zero, F(l, 21) = 302.8,/? <.01. In addition, the difference in the size of the workload reduction across visibility levels was 
reliable F(l, 21) = 7.16, p <.02. No effects involving pilot group approached significance. We conclude from these results that both 
the HUD and the EMM produced large reductions in pilot workload, and these reductions were larger under low visibility than under 
moderate visibility. Compared to the workload reduction produced by each navigation aid separately, no additional reduction was 
seen when the aids were jointly available. 

Table 4: Workload ratings as a function of Pilot Group and Visibility 


Visibility 



RVR 700 ft 

RVR 1400 ft 

Baseline 

7.0 

6.3 

EMM 

4.1 

4.0 

Difference 

2.9 

2.3 

Baseline 

6.3 

5.2 

HUD 

3.2 

3.2 

Difference 

3.1 

2.0 

Baseline 

5.3 

4.8 

EMM + HUD 

2.8 

2.6 

Difference 

2.6 

2.2 


SUMMARY AND CONCLUSIONS 

At present, surface operations are one of the most inefficient areas of operation in the national airspace system. Much of the 
inefficiency can be traced to the inherent difficulty of the ground navigation task. However, recent simulation work has shown that 
support for navigation awareness, in the form of EMM’s, produce substantial improvements in ground taxi performance. The present 



work investigated whether performance could be further enhanced by HUD symbology that combines limited support for navigation 
awareness with explicit cues for local guidance. 

The results of our simulation can be summarized as follows. Relative to the baseline condition, both the EMM and the HUD taxi 
symbology yielded a substantial reduction in route completion time. This reduction can be traced to three sources. First, both displays 
virtually eliminated navigation errors, even in low visibility. In the baseline condition, by contrast, navigation errors were common. 
Second, both displays saved approximately half a minute in route planning time. Third, they both produced an increase in forward taxi 
speed. Under low visibility, the speed increase was much larger when the EMM was accompanied by the HUD symbology than when 
it was not. Finally, complementing these empirical results, both displays produced a large reduction in the workload associated with 
the ground taxi task. 

In an attempt to make sense of this pattern of results, we return to the parsing of the ground taxi task into local guidance and 
navigation awareness components. Previous work [1] suggests that the cognitive processes involved in maintaining navigation 
awareness are highly effortful. Because they both provide route guidance information, the EMM and the HUD remove the need to 
engage in these forms of processing. Thus, both aids produced similar reductions in navigation error and route planning time. 

Unlike navigation awareness, local guidance is largely a set of direct feedback loops between perception and action [12], involving 
little in the way of effortful cognitive processing. The fact that local guidance is relatively effortless may explain why the HUD did 
not provide further reductions in workload, over and above the EMM. On the other hand, forward speed is largely controlled by visual 
cues such as edge rate [20], which are supplied by the HUD symbology but not the EMM. This provides a natural explanation for why 
taxi speed was more sensitive to the presence of the HUD than the EMM. As for the finding that sensitivity to the HUD symbology 
increased as visibility decreased, we note that, in moderate visibility, visual cues for local guidance are available in the out-the- 
window scene. Thus, providing the cues redundantly on the HUD yielded only a modest benefit, over and above the EMM alone. In 
low visibility, however, the visual cues that normally control local guidance are either unavailable or severely degraded [5], Since the 
HUD was now the sole source of undegraded local guidance cues, the relative advantage of an EMM plus a HUD over an EMM alone 
was greater. 

This account is, of course, only tentative. In particular, it is not clear how much of the HUD-related speed increase is actually due 
to the availability of local guidance cues. The HUD symbology also supports aspects of the navigation task that skirt the boundary 
between local guidance and navigation awareness [5], For example, the virtual path provides information about the distance to the 
next turn, and how rapidly the aircraft is approaching it. Pilots may well have used this information to maximize their forward speed 
through long, straight sections of a route, since they know precisely how far they have to travel before slowing down. A better 
understanding of the precise source of the forward speed benefit with the HUD symbology awaits further research. 

PRACTICAL IMPLICATIONS 

The results of the present simulation suggest that adding advanced navigation aids hold the promise of substantially increasing the 
efficiency of surface operations, particularly under reduced visibility conditions. In large part, this advantage is due to the fact that 
these aids virtually eliminated navigation errors. In addition, the results indicate that under reduced visibility, a display that provides 
local guidance cues can substantially increase forward taxi speed, over and above the increase found with navigation awareness 
displays (i.e., an EMM). Indeed, under low visibility, the combination of few or no navigation errors and relatively high taxi speed 
reduced taxi time by fully three min in the EMM + HUD group. This represents a 40% improvement over today's taxiing environment 
(i.e., the baseline condition). Assuming that these benefits translate to real world operations, they are large enough to impact both 
airline schedules and fuel costs significantly. 

It is important to realize, however, that these benefits require both forms of navigation display. The EMM abolishes navigation 
errors, but provides only a modest increase in taxi speed. The HUD yields more of a speed benefit, as well as abolishing navigation 
errors, but all pilots in the simulation thought that the EMM provided a better level of situation (navigation) awareness. When given 
the choice of navigating with only an EMM, only a HUD, or both, all 24 participants said they would prefer to see both displays in the 
flight deck. 

Finally, considering the performance benefits obtained in the present simulation, we want to point out that the EMM and the HUD 
represent only a "first pass" at display design. More advanced versions of both displays are currently under development [5][1 1] and 
will soon be augmented by a head-up auditory warning system for traffic conflicts [5], Thus, although the benefits realized in the 
present simulation are impressive, further improvements can be expected from future designs. 
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